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ABSTRACT 

Objective: Computerised drug alerts are expected to 
reduce patients' risk of adverse drug events. However, 
physicians over-ride most drug alerts, because they 
believe that the benefit exceeds the risk. The purpose 
of this study was to determine the drug alert, patient 
and physician characteristics associated with the: 

(1) occurrence of psychotropic drug alerts for elderly 
patients and the (2) response to these alerts by their 
primary care physicians. 

Setting: Primary care, Quebec, Canada. 

Design: Prospective cohort study. 

Participants: Sixty-one physicians using an electronic 

prescribing and drug alert decision-support system in 

their practice, and 3413 elderly patients using 

psychotropic drugs. 

Primary and secondary measures: Psychotropic 
drug class, alert severity, patient risk for fall injuries 
and physician experience, practice volume and 
computer use were evaluated in relationship to the 
likelihood of having: (1) a psychotropic drug alert, 

(2) the prescription revised in response to an alert. 
Cluster-adjusted alternating logistic regression was 
used to assess multilevel predictors of alert occurrence 
and response. 

Results: In total 13 080 psychotropic drug alerts 
were generated in 8931 visits. Alerts were more 
likely to be generated for male patients at higher 
risk of fall-related injury and for physicians who 
established the highest alert threshold. In 9.9% of 
alerts seen, the prescription was revised. The 
highest revision rate was for antipsychotic alerts 
(22.6%). Physicians were more likely to revise 
prescriptions for severe alerts (OR 2.03; 95%CI 
1.39 to 2.98), if patients had cognitive impairment 
(OR 1.95; 95%CI 1.13 to 3.36), and if they made 
more visits to their physician (OR 1.05 per 5 visits; 
95%CI 1 to 1.09). 

Conclusions: Physicians view and respond to a 
small proportion of alerts, mainly for higher-risk 
patients. To reduce the risk of psychotropic 
drug-related fall injuries, a new generation of 
evidence-based drug alerts should be developed. 



ARTICLE SUMMARY 



Article focus 

■ Computerised decision support is expected to 
reduce prescribing errors that may increase the 
risk of adverse events but most alerts are 
ignored. 

■ We assessed whether psychotropic drug alerts 
were more likely to be generated for higher-risk 
patients, and for physicians in relationship to their 
demographic and practice characteristics. We also 
assessed whether these characteristics were asso- 
ciated with the likelihood that a prescription would 
be revised in relationship to an alert. 

Key messages 

■ Commercial drug decision-support systems gen 
erate many alerts for psychotropic drugs, 97% 
of which are of moderate or serious severity (not 
severe), and the majority of these alerts will be 
ignored by primary care physicians. 

■ Patients at greater risk of fall-related injuries are 
more likely to have psychotropic drug alerts, as 
are male physicians, and physicians who screen 
out the majority of alerts. 

■ Physicians are more likely to change a prescrip- 
tion in response to an alert if the patient is a fre- 
quent visitor at the clinic and if they have 
cognitive impairment. 

Strengths and limitations of this study 

■ This is the first study to use empirically generated 
audit trails within a computerised medical record 
to simultaneously evaluate the drug, patient and 
physician characteristics that are associated with 
the likelihood of having a psychotropic drug alert 
generated by a commercial drug alert system, as 
well as the physicians' response to an alert, in a 
large cohort of primary care physicians and their 
elderly patients. 

■ We were unable to assess all possible alert 
related predictors of physician response (eg 
type of alert) as the majority of physicians 
elected to view only the severe alerts. 
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INTRODUCTION 

Psychotropic medication use is associated with an 
increased risk of adverse drug events such as fall-related 
injuries in older persons. 1-4 Fall-related injuries among 
older adults are a significant source of morbidity, mortality 
and healthcare costs. In the USA, about a third of adults 
65 years and older experience a fall annually, resulting in 
US$19 billions of healthcare service use. 5 6 Half of the 
adverse drug events associated with fall-related injuries in 
older persons are considered preventable and a majority 
of errors that contribute to preventable adverse drug 
events occur at the time of prescribing. 7 

Improving the management of psychotropic medica- 
tions has been identified as a potentially cost-effective 
means by which to reduce fall-related injuries in older 
persons. 7 8 Computerised prescribing systems integrated 
with clinical decision support systems could serve as a 
means to more effectively prescribe psychotropic medica- 
tions because these systems have been found to prevent 
medication errors by providing physicians with alerts at 
the point of prescribing. 9-11 However, evidence is mount- 
ing that physicians override the majority of computerised 
drug alerts they receive from such systems — reported at 
upwards of 90% in some studies 4 — which may limit the 
utility of such systems in improving medication manage- 
ment of psychotropic medications in older adults. 

Physicians' high rates of over-riding computerised drug 
alerts may be associated with 'alert fatigue,' which has been 
theorised to occur when a high volume of drug alerts of 
limited clinical relevance are generated. 12-14 Interventions 
have been developed to address 'alert fatigue' by modifying 
how alerts are presented and viewed, including reducing 
the volume of clinically irrelevant alerts through an expert 
panel review 15 or by providing alerts on-demand to physi- 
cians. 16 However, these interventions have met with varying 
levels of success. Furthermore, evaluations of physicians' 
response to alerts suggest that a complex set of factors may 
shape physicians' usage of these systems including features 
of the computerised drug alert system and alerts 15 17 18 the 
complexity of patients' health needs 19 and physician char- 

. . , . . 19 

actenstics and practice environment. 

Identifying the multilevel determinants of response to 
alerts may help refine the content and delivery of alerts 
in a manner that is better attuned to physicians' practice 
and patients' needs. 

We conducted a study to identify factors associated 
with physicians' response to alerts for older adults on 
psychotropic medications. We also sought to characterise 
the key steps leading to a physician's response to alerts, 
including factors associated with the generation and 
viewing of the alert, as these steps may be linked to 
whether physicians subsequently respond to an alert. 



METHODS 

Study design and population 

We conducted a prospective cohort study with 61 
primary care physicians, and all of their patients aged 



65 years of age or older who were prescribed psycho- 
tropic drugs (n=3413). All computerised drug alerts 
generated for these patients between 1 January and 
31 December 2007 that involved their psychotropic med- 
ications were retrieved. This study was approved by the 
McGill University Institutional Review Board. 

Electronic prescribing system 

All physicians used a bilingual (English/French) elec- 
tronic health record and drug management system 
(MOXXI) that is, integrated with the provincial health 
insurance agency (Regie de 1' assurance maladie du 
Quebec, RAMQ) . The RAMQ provides daily downloads 
of all new medical services and prescriptions for all con- 
senting patients in the practice from its population- 
based health insurance databases, enabling new health 
problems, procedures, emergency department visits, 
hospitalisations and medications to be automatically 
updated in the patient's health record. The MOXXI 
system includes a health problem list, an electronic 
prescriber, information on all medications dispensed 
including the drug, strength, quantity, duration and pre- 
scribing physician, and automated alerts for potential 
prescribing problems. 20 

The automated prescribing alerts and drug knowledge 
database, maintained by Vigilance Sante (Montreal, 
Canada), generates alerts using information from the 
patient's current medication list and new electronic pre- 
scriptions for all major types of prescribing problems, 
including excess dose, drug-allergy, drug-drug, drug- 
disease, drug-age contraindications and therapeutic 
duplication (www.vigilance.ca). The drug knowledge 
base classifies all types of alerts by the level of clinical 
importance on a three-level severity scale, with level 1 
alerts defined as 'severe alerts — avoid completely', 
level 2 alerts defined as 'serious alerts, avoid if possible' 
and level 3 alerts as 'moderate alerts, use with caution'. 
Alerts are presented in an interruptive manner, forcing 
physicians to either change the prescription to remove 
the alert, or enter a reason for ignoring the alert before 
being able to print the prescription (see the online 
appendix for screenshots) . Physicians may view a 
detailed description of the interactions or contraindica- 
tions identified in the alert by clicking in a detail panel. 
In addition, physicians have the option to suppress and 
not view serious and/or moderate alerts (eg, level 2 
and/ or 3) as well as individual drug alerts that he or she 
disagrees with at the time the alert is generated. In add- 
ition, all alerts, current and previously suppressed, can 
be viewed by the physician in a non-interruptive manner 
by accessing the alert history section of the application. 

Data sources 

Audit trails in the MOXXI database were used to 
measure physicians' usage of the system, including date 
and time of use, screens viewed and functions used, 
prescriptions written, alerts generated, alerts seen by the 
physician (physician alert level preference) and the 
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physician's response to alerts he or she viewed. 
Physicians' characteristics and e-prescribing experience 
was assessed at the completion of training using question- 
naire and standardised e-prescribing skill assessment. 11 

Patient and practice characteristics were assessed using 
data from Quebec's universal public health insurance 
program (RAMQ). For each consenting patient and 
physician, data were available from the beneficiary, 
prescription and medical services claims databases to 
measure beneficiary age, sex and neighbourhood mea- 
sures of socio-economic status as well as all medical 
visits, hospitalisations, prescriptions and health problems 
and injuries recorded in the year prior to enrolment 
and throughout the study period. 21 The accuracy of 
these data have been validated in prior studies. 22-28 

Outcomes 

Alert generated 

Using data from the MOXXI database, we determined, 
at each patient visit, whether one or more psychotropic 
drug alerts was generated. An alert could be generated 
on the basis of existing active medication (the patient's 
profile of dispensed medication) and/or a newly pre- 
scribed psychotropic medication. To evaluate patient 
and physician characteristics associated with the gener- 
ation of alerts, each visit was classified as a visit with or 
without a psychotropic drug alert. 

Alert medication revised 

For each psychoactive drug alert that was generated at a 
visit, we first assessed whether the physician saw the alert 
as a function of the physician's 'alert level setting'. 
Among alerts that were seen, we assessed whether the 
physician made an effort to modify, stop or change 
the dose of one or more of the drugs implicated in the 
alert, or alternately over-rode the alert in which case 
they were required to record the reason for doing so. 

Multilevel predictors 

Physician-level predictors 

Physician demographic characteristics examined included 
gender and practice experience, defined as the number 
of years since graduation from medical school. Both 
were measured at enrolment as these characteristics are 
associated with prescribing behaviour and may be pre- 
dictive of alert generation and response. Practice character- 
istics such as practice volume and the proportion of 
elderly in the practice are also associated with prescrib- 
ing behaviour. 29 Daily practice volume was calculated 
based on medical service billing claims made by each 
physician during 2006, calculated by counting the 
number of visits the physician billed for that year and 
dividing by the number of workdays with one or more 
billings. The percentage of patients 65 years and older 
was measured as the number of patients 65 years or 
older that the physician billed a service for in 2006, 
divided by the total number of patients the physician 
billed a service for in 2006. Experience and skills related to 



electronic prescribing was also measured as a factor in alert 
response as it may influence the speed and confidence 
in modifying prescriptions in response to an alert. The 
electronic prescription rate was measured by calculating 
the number of electronic prescriptions (e-RXs) per 100 
visits, and the electronic prescription speed was defined 
as the time (in minutes) to write three standardised pre- 
scriptions electronically. Lastly, the physician-selected 
alert-level setting was measured as a possible determinant 
of alert response. Physicians could choose to view severe 
alerts only (level 1 alerts), view severe and serious 
alerts (levels 1 and 2 alerts), or view all alerts (levels 1, 2 
and 3 alerts) as 'interruptive alerts' during the prescrib- 
ing process. 

Patient-level predictors 

Five patient attributes were measured as factors that may 
influence the likelihood of alerts as well as physician 
response to alerts. Patient age was measured as of 
1 January 2007, and patient gender was identified. In 
addition, patient risk factors for fall-related injuries were 
measured from ICD9 diagnostic and procedure codes in 
their medical service claims from the years 2005 and 
2006, using a previously validated set of codes for each 

21 24 30 31 

risk factor. Risk factors measured included the 

presence of a prior fall-related injury, gait and balance 
problems, lower-extremity weakness, and cognitive 
impairment. We also assessed each patient's medication 
complexity by counting the number of active medica- 
tions (prescribed or dispensed, including psychotropic 
medications) at each visit. Lastly, we measured visit fre- 
quency as a reflection of the opportunity to modify treat- 
ment, defined as the number of ambulatory care visits 
made in the year prior to the visit date. 

Alert-level predictors 

The attributes of each alert was characterised by type of 
alert, severity and therapeutic class. Type of alert was clas- 
sified into one of nine mutually exclusive categories: 
drug-disease contraindication, cumulative side effects, 
drug-age contraindication, therapy duplication, drug 
interaction, excess dose, renal dose adjustment, drug or 
potential cross allergy and other. Alert severity was classi- 
fied as moderate, serious or severe using the classifica- 
tion established by the vendor (www.vigilance.ca). 
Therapeutic class of drug(s) causing the alert included anti- 
depressants, antipsychotics, anticonvulsants, benzodiaze- 
pines, opioids and other psychoactive classes. 

Analysis 

To estimate the relationship between the alert, patient 
and physician predictors and (1) alert generation and 
(2) response to alerts we used alternating logistic regres- 
sion. Alternating logistic regression models provide a 
method of deriving unbiased estimates of association as 
it permits non-independence of observations with more 
than one level of clustering to be managed within the 
context of a dichotomous outcome. For the outcome, 
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alert generated (yes/no), the unit of analysis was visit 
and the outcome was modelled with two levels of cluster- 
ing: visits within patients, and patients within physician. 
For the outcome, alert revised (yes/no), the unit of ana- 
lysis was alert and the outcome was modelled with two 
levels of clustering; alerts within patient and patients 
within physician. For the generation of alerts outcome, 
sample size was sufficient to include all variables in the 
multivariate model using alternating logistic regression. 
However, for the revision of alerts outcome the sample 
size of alerts that were actually seen by physicians was 
not sufficient to include all variables in the multivariate 
model and have it converge. Therefore, only variables 
found to have a bivariate association with a level of statis- 
tical significance of p=0.20 or less were included in the 
final model predicting revision of alerts. 

RESULTS 

Overall, 44% of physicians were female. On average, 
physicians had 23.4 years of practice experience, 21% 
(SD 11.7%) of patients in the physicians' practice popu- 
lation were 65 years of age and older, 21 patients were 
seen per day and 53.7 electronic prescriptions were 
written per 100 visits (table 1). Physicians varied in 
computer-based prescribing proficiency with a range in 
the time required to write three standardised prescrip- 
tions of 2.8-10.4 min. Most physicians (82%) set the 
drug alert preference to view only severe alerts. Eligible 
patients had a mean age of 75.5 years, 9.4 active drugs 
and 12.2 visits. In total 67.5% were female, and up to 
24.1% had at least one risk factor for fall-related injuries, 
the most common being gait and balance problems 
(table 1). 

In 2007, 13 080 psychotropic drug alerts were gener- 
ated in the 8931 visits made by study patients; an average 
of 1.5 psychotropic drug alerts per visit. The majority of 
alerts were drug-disease contraindications (42.9%) fol- 
lowed by cumulative side effects (23.8%; table 2). Most 
alerts were of serious (52%) or moderate (45.1%) clin- 
ical importance. Antidepressants (45%) and benzodiaze- 
pines (30.8%) were most common drugs implicated. 
Only a minority of the alerts generated (6.2%) were 
seen because the majority of physicians had set their 
preferences to see only severe alerts. As a result, alert 
types most likely to be seen were for excess dose 
(69.8%), drug or potential cross allergies (29%) and 
drug-age contraindications (18.3%). Among those that 
were seen, 9.9% were revised, with the highest revision 
rate being for excess dose (14.8%), renal dose adjust- 
ment (16.7%), severe alerts (16.2%) and for antipsycho- 
tics (22.6%). 

Factors associated with alert occurrence 

Alerts were more likely to be generated at a visit for 
patients who had a prior fall-related injury (OR 1.44; 
95% CI 1.03 to 2.01; table 3). Female patients after 
adjusting for greater visit frequency were less likely than 



Table 1 Physician and patient characteristics 




Physicians (n=61) 



Demographic characteristics 


N 


% 


Female 


27 


44 




Mean 


SD 


Practice experience (years) 


23.4 


7.1 


Practice characteristics 






Daily practice volume 


21.2 


7.4 


Percent of patients 65 years 


21 


11.7 


and older 






Experience and skills related to electronic prescribing 


Electronic prescription rate 


53.7 


45.7 


(e-RXs per 100 visits) 






Electronic prescription speed 


5.5 


1.7 


(minutes per 3 e-RXs) (Range) 


(2.8-10.4) 




Alert level setting 


N 


% 


View severe alerts only 


50 


82 


View serious and severe 


9 


14.8 


alerts only 






View all alerts (moderate, serious 


2 


3.2 


and severe) 






Patients (n=3413) 


Mean 


SD 


Age 


75.5 


6.6 




N 


% 


Female 


2303 


67.5 


Risk factors for fall-related injuries (in prior 2 years) 




Prior fall-related injury 


147 


4.3 


Gait and balance problems 


822 


24.1 


Lower-extremity weakness 


272 


8 


Cognitive impairment 


288 


8.4 


Visits (n=8931) 


Mean 


SD 


Number of ambulatory care visits 


12.2 


9.1 


(in year prior to visit) 






Number of active medications 


9.4 


4.2 


(at visit) 







males to have a psychotropic drug alert generated 
during the visit (OR 0.85; 95% CI 0.75 to 0.98). Female 
physicians (OR 0.61; 95% CI 0.42 to 0.91), and physi- 
cians who saw more patients per day (OR per 5 patient 
increase: 0.89, 95% CI 0.79 to 0.99) were less likely to 
have psychotropic drug alerts, whereas physicians who 
set their alert preferences to view only the severe alerts 
were 99% (95% CI 1.04 to 3.81) more likely to have a 
psychotropic drug alert generated for their patients. 

Factors associated with alert response 

As only 80 alerts were revised, we were unable to include 
all potential predictors of alert response in a multivari- 
ate alternating logistic regression model. Therefore, we 
included only predictors with a p value <0.20 in the 
bivariate association (table 4). Severe alerts were twice as 
likely to be revised when compared to serious or moder- 
ate alerts (OR 2.03; 95% CI 1.39 to 2.98). Alerts for 
renal dose adjustment were the most likely type of alert 
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Table 2 Characteristics of alerts by stage in alert generation, viewing and revision process 



Alerts generated 
(n=13080 in 8931 visits) 

n % 



Alerts seen* (n=807) 



Percentage seen 
(of alerts generated) 



Alerts revised (n=80) 

Percentage revised 
n (of alerts seen) 



Type of alert 



Drug-disease 


5606 


42.9 


144 


2.6 


Dntraindication 










Cumulative side effects 


3109 


23.8 


112 


3.6 


Drug-age contraindication 


1600 


12.2 


293 


18.3 


Therapy duplication 


1345 


10.3 


102 


7.6 


Drug interaction 


1087 


8.3 


43 


4 


Excess dose 


126 


1 


88 


69.8 


Renal dose adjustment 


122 


0.9 


12 


9.8 


Drug or potential cross allergy 


31 


0.2 


9 


29 


Other 


54 


0.4 


4 


7.4 



Severity of alert 
Severe 
Serious 
Moderate 



387 
6797 
5896 



Therapeutic class of drug(s) causing alertt 

Antidepressants 5886 

Antipsychotics 1531 
Anticonvulsants 990 

Benzodiazepines 4033 

Opioids 1781 

Other psychoactive classes 2725 



3 
52 
45.1 



45 
11.7 
7.6 
30.8 
13.6 
21 



315 
462 
30 



239 
62 
42 

250 
56 

272 



81.4 
6.8 
0.5 



4.1 
4 

4.2 
6.2 
3.1 
10 



\ D 


111 
i i . i 


8 


7.1 


OU 


mo 


8 


7.8 


i 




I o 


1/1 o 
1 H-.O 


o 
c. 


1 ft 7 
I D. / 


I 


111 
I I . I 


I 




51 


16.2 


29 


8.8 


0 


0 


9 


3.8 


14 


22.6 


4 


9.5 


33 


13.2 


5 


8.9 


24 


8.8 



*AII alerts generated by a patient's profile of active or newly prescribed medication are available for a physician to view, and those shown in 
an 'interruptive' manner are a function of the alert setting preferences that is, set by the physician in accordance with severity (mild, moderate 
and severe). 

tThe sum of the alerts by drug class may exceed the total as an alert may be associated with more than one drug class. 



to be revised (OR 1.64; 95% CI 1.09 to 2.46), with a sig- 
nificantly higher likelihood of revision than drug-age 
contraindications (reference group) and drug interac- 
tions (OR 0.49; 95% CI 0.22 to 1.08). Drug alerts for 
patients with cognitive impairment were 95% more 
likely to be revised than those with no cognitive pro- 
blems (OR 1.95; 95% CI 1.13 to 3.36), and the likeli- 
hood that an alert would be revised increased for 
patients who made more visits to their physician (OR 
1.05 per 5 visit increase; 95% CI 1.004 to 1.09). 
Physician sex, practice experience, practice characteris- 
tics, computer skill and alert level setting were not asso- 
ciated with the likelihood of responding to an alert. 

DISCUSSION 

This is the first study to employ empirically generated 
audit trails with a computerised medical record to inves- 
tigate the alert, patient and physician characteristics 
associated with both the generation of psychotropic 
alerts from a commercially available drug alert system 
and physician response in primary care. We found that 
psychotropic drug alerts were common, with 1.5 alerts 
occurring per visit, that the majority (97%) were 



moderate or serious alerts (not severe alerts) and that 
physicians saw a minority (6.1%) of these alerts because 
they (82%) set their threshold for viewing alerts to see 
only the severe alerts. Patients at higher risk of fall- 
related injuries were more likely to have alerts for their 
psychotropic drugs, and were also more likely to have 
medications revised in response to an alert. Alerts were 
more likely to be generated for patients' psychotropic 
medications if their physician was male and had estab- 
lished an alert threshold to view only the severe alerts. 
Additionally, the severity and type of the alert were asso- 
ciated with a physician's response to an alert. 

Our findings are coherent with prior research, 
as well as a recent Delphi assessment of important attri- 
butes of drug alert systems to improve clinical utility. 35 
Alerts are considered clinically useful when graded by 
severity, when patient risk is considered, and when the 
probability of an adverse event is significant. Physicians 
in this study responded in a similar manner; preferen- 
tially revising severe alerts and in patients who were at 
higher risk of an adverse medication-related event. Both 
Isaac and Grizzle noted a similar pattern. 17 34 Physicians 
were more likely to respond to the most severe alerts, 
and for patients who were being newly exposed to a 
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Table 3 Patient and physician level predictors of alert generation 




Multivariate logistic regression 
Descriptive statistics analysis* 





No alert 


Alert 










Number of visits=8931 


(n=3863) 


(n=5068) 


OR 


95% CI 


p Value 


Patient 
















Demographic characteristics 


n 


% 


n 


% 








Female 


2730 


70.7 


3547 


70 


0.85 


0.75 to 0.98 


0.02 


Male 


1133 


29.3 


1521 


30 


Reference 


Reference 


Reference 




ivicdi i 


9n 

ou 


Moan 


ou 








Age (per year) 


75.7 


6.5 


75.9 


6.6 


0.99 


0.98 to 1 


0.17 


Risk factors for fall-related injuries 


PI 


% 


pi 


% 








Fall-related injury 


ou^ 


7 ft 


oou 


114 

I I .H- 


1.44 


1 .03 to 2.01 


0.04 


Gait and balance problems 


938 


24.3 


1447 


28.6 


1.15 


0.99 to 1.33 


0.06 


Lower-extremity weakness 


317 


8.2 


435 


8.6 


0.95 


0.75 to 1.21 


0.68 


Cognitive impairment 


193 


5 


208 


4.1 


0.91 


0.64 to 1 .29 


0.59 




ivicdi i 


ou 


Moan 
ivitJdi i 


o vj 








Number of ambulatory care visits (year prior to visit) 


1 1 .2 


7.7 


12.9 


10 


1.08 


1.05 to 1.12 


<0.01 


(OR per 5 visit increase) 
















Number of active medications (at visit) 


o.o 


^ ft 
o.o 


1 n ^ 

I u.o 


A A 


1.12 


1.10 to 1.14 


<0.01 


Physician 
















Demographic characteristics 




% 


PI 


% 








Female 


141 1 
i *+ i i 


OU.J 


14RP 

I l -T\JC. 




0.61 


0.42 to 0.91 


0.01 


Male 


2452 


UO.J 




71 .2 


Reference 


Reference 


Reference 




Mean 


SD 


Mean 


SD 








Practice experience (OR per 5 year increase) 


26.2 


5.4 


26.5 


5.7 


0.89 


0.76 to 1.03 


0.12 


Practice characteristics 
















Daily practice volume (OR per 5 patient increase) 


23.1 


6.7 


21.7 


7 


0.89 


0.79 to 0.99 


0.03 


Percent of patients 65 years and older (OR per 


25.7 


10.8 


25.5 


11.4 


1 


0.89 to 1.11 


0.96 


10% increase) 
















Experience and skills related to electronic prescribing 
















Electronic prescription speed (minutes per 3 e-RXs) 


5.3 


1.6 


5.6 


1.6 


1.01 


0.93 to 1.10 


0.78 


(per minute increase) 
















Electronic prescription rate (e-RXs per 100 visits) 


25.7 


13 


27.6 


12.3 


1.06 


0.96 to 1.17 


0.28 


(OR per 10 e-RX increase) 
















Alert level setting 
















View severe alerts only 


3143 


41 


4527 


59 


1.99 


1 .04 to 3.81 


0.04 


View serious and severe alerts only, or view all alerts 


885 


44.9 


1085 


55.1 


Reference 


Reference 


Reference 



*The dataset comprised 8931 visits clustered within 3413 patients clustered within 61 physicians. Alternating logistic regression was used to 
account for clustering (visits within patients and patients within providers). 



drug or drug combination, where the risk would be 
higher than those who had already demonstrated that 
they could tolerate the medication. These findings as a 
group suggest that alerts will be most useful if graded by 
clinical importance, and if patient-specific estimates of 
risk are calculated and presented, so that only the most 
clinically relevant alerts could be presented. 

However, one of the greatest barriers to developing 
more clinically useful alert systems is the paucity of 
empirical evidence of the risk associated with the vast 
majority of alerts. Most current alert systems, both com- 
mercial and home-grown, base the grading of alerts 
as well as the probability of harm on lowest level of 
evidence — expert opinion, 36-38 although there are 
notable recent attempts 39 to grade alerts by the strength 
of the evidence. To make substantial gains in improving 



drug safety, and reducing the risk of adverse drug events 
with computerised decision support, real-world evidence 
needs to be generated on the risks associated with drug 
alerts, and this evidence needs to be incorporated into a 
new generation of alert systems. For example, with the 
increasing use of electronic medical records, an oppor- 
tunity exists to use clinical and administrative data to 
monitor adverse drug events and model the actual risks 
of adverse events associated with prescribed medications. 
Moreover, priority should be placed on drugs and drug 
combinations that have the highest population attribut- 
able risk, which is a function of the prevalence of use, 
magnitude of the risk and severity of adverse effects. 
This would likely include the most prevalent classes of 
medication — cardiovascular drugs, antibiotics, psycho- 
tropic medications, anti-inflammatories and analgesics, 
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Table 4 Factors associated with revising a prescription in response to an alert 



Descriptive statistics Regression analysis* 
Over-ride Revise 



Bivariate 



Multivariate 



Alerts seen=807 



(n=727) (n=80) 



OR 



95% CI 



p Value OR 



95% CI 



p Value 



Alert characteristics 
Severity of alert 



% 



Most severe 


264 


36 


51 


64 


2.21 


-4 ACS J. CS CS C\ 

1 .48 to 3.29 


<0.01 


Moderate or less 


463 


64 


29 


36 


Reference 


Reference 


Refer* 


severe 
















Type of alert 
















Drug-age 


263 


36 


30 


38 


Ret 


Ret 


Ret 


contraindication 
















Cumulative side 


104 


14 


8 


10 


0.97 


cs ^cs j_ u cs cs 

0.73 to 1 .30 


0.86 


effects 
















Drug— disease 


128 


18 


16 


20 


0.94 


0.64 to 1 .38 


0.75 


contraindication 
















Therapy duplication 


94 


13 


8 


10 


0.97 


0.57 to 1 .67 


0.92 


Drug interaction 


42 


6 


1 


1 


0.67 


0.49 to 0.93 


0.02 


Excess dose 


75 


10 


13 


16 


0.98 


0.50 to 1 .92 


0.95 


Renal dose 


10 


1 


2 


3 


1.55 


1.07 to 2.25 


0.02 


adjustment 
















Drug or potential 


8 


1 


1 


1 


0.93 


0.49 to 1 .74 


0.81 


cross allergy 
















Other 


3 


1 


1 


1 


1.42 


0.67 to 3.02 


0.36 


Patient characteristics 
















Female 


475 


65 


50 


63 


1.02 


0.70 to 1.47 


.94 


Male 


252 


35 


30 


38 


Ref 


Ref 


Ref 




Mean 


SD 


Mean 


SD 








Age (years) 


74.5 


6.2 


74.5 


6.5 


1.02 


0.99 to 1.04 


0.25 


Risk factors for 


n 


% 


n 


% 








fall-related injuries 
















Fall-related Injury 


32 


4.4 


5 


6.3 


0.89 


0.35 to 2.26 


0.80 


Gait and balance 


212 


29.2 


29 


36.3 


1.18 


0.78 to 1.78 


0.44 


problems 
















Lower extremity 


61 


8.4 


7 


8.8 


1.31 


0.68 to 2.51 


0.42 


weakness 
















Cognitive 


103 


14.2 


12 


15 


1.67 


1.03 to 2.70 


0.04 



2.03 1.39 to 2.98 

Reference Reference 



10.8 4.9 10.9 4.8 1.01 



% 



% 



impairment 

Number of ambulatory 16.2 23.2 18.6 34.1 1.02 
visits (year prior to 
visit) (OR per 5 visit 
increase) 
Number of active 
medications (at visit) 
Physician characteristics 
Demographic 
characteristics: 

Female 

Male 



per 5 year increase) 

Practice 

Characteristics 
Daily practice 
volume (OR per 5 
patient increase) 



0.99 to 1.06 0.19 



0.97 to 1.06 0.52 



119 


16 


25 


31 


1.63 


0.73 to 3.65 


608 


84 


55 


69 


Reference 


Reference 


Mean 


SD 


Mean 


SD 






24.2 


5.4 


24.7 


6.7 


0.96 


0.60 to 1.55 



Ref 

1.06 

0.74 

1.14 
0.49 
0.67 
1.64 

0.66 

1.41 



1.95 
1.05 



20.4 6.1 19.3 6.8 088 



0.65 to 1.02 0.38 



Ref 

0.70 to 1 .60 

0.44 to 1 .23 

0.47 to 2.79 
0.22 to 1.08 
0.34 to 1.34 
1 .09 to 2.46 

0.37 to 1.19 

0.80 to 2.49 



<0.01 
Reference 



Ref 

0.79 

0.24 

0.77 
0.08 
0.26 
0.02 



0.17 
0.24 



1.13 to 3.36 0.02 

1 .004 to 0.03 
1.09 



Continued 
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Table 4 Continued 




Descriptive statistics 


Regression analysis* 








Over-ride Revise 


Bivariate 


Multivariate 




Alerts seen=807 


(n=727) (n=80) 


OR 95% CI 


p Value OR 95% CI 


p Value 



Percentage of 
patients 65 years 
and older (OR per 
10% increase) 

Experience and skills 

related to electronic 

prescribing 
Electronic 
prescription speed 
(minutes per 3 
e-RXs) (per minute 
increase) 
Electronic 
prescription rate 
(e-RXs per 100 
visits) (per 10 e-RX 
increase) 

Alert level setting 
View severe alerts 
only 

View serious and 
severe alerts only, 
or view all alerts 



19.5 9.1 19 



n 

323 



8.3 0.83 



0.44 to 1 .56 0.56 



5.5 1.6 5.8 1.8 1.18 



31.4 12.4 28.8 15.9 0.84 



0.90 to 1.55 0.23 



0.65 to 1.10 0.20 



0.86 



0.66 to 1.13 0.27 



% n 
44.4 48 



% 
60 



1.80 



0.91 to 3.57 0.09 



1.11 



0.54 to 2.28 0.78 



404 55.6 32 40 Reference Reference Reference Reference Reference Reference 



*The above dataset contained 807 alerts clustered within 303 patients, who were clustered within 47 physicians. For patients clustered within 
physicians, minimum to maximum cluster size was 1 to 141, respectively. Alternating logistic regression was used to account for two levels of 
clustering (alerts within patients; patients within physicians). 



as well as those with established high risks such as 
anticoagulants. 

This is the first study to document that physicians who 
are most likely to generate alerts are also more likely to 
establish the highest threshold for viewing alerts. This 
finding, while new, fits with prior observations on the 
high frequency of overriding computerised prescribing 
alerts by physicians. 12 15 16 19 Alert fatigue created by 
too many drug alerts is a common complaint, 17 34 and 
methods of over-riding, or suppressing alerts are highly 
valued, particularly when alerts are presented in an 
interruptive manner. However, if physicians with a high 
volume of drug alerts are more likely to have a higher 
rate of adverse drug events in their patient population, 
then facilitating alert over-rides defeats the purpose of 
implementing computerised decision support in the first 
place. It has been assumed that reducing the sheer 
number of alerts to the most serious problems will solve 
the problem of alert fatigue and alert over-rides. But this 
assumption may not be correct. A growing body of litera- 
ture in educational psychology suggests that physicians 
who have the highest prevalence of quality and safety 
problems in practice may also be most likely to over-rate 
the quality of their performance. 40 If these factors influ- 
ence alert response, then the physicians who are most 
likely to generate severe drug alerts may be least likely to 
respond to them. This possibility needs to be assessed 



in future research, as more comprehensive feedback, 
such as comparative benchmarking for quality of drug 
management as well as patients outcomes, may need 
to be included with drug alerts to alter physician's self- 
assessment of performance so that safety and quality 
problems will be addressed. 41 

Our results need to be considered in light of several 
limitations. First, the number of alerts that were actually 
seen by physicians and revised was small, and we had 
insufficient power to identify factors that may have 
increased the likelihood of responding to an alert by 
50% or less. Our sample is limited to primary care physi- 
cians and psychotropic drug alerts. The generalisability 
of these findings to alerts related to other classes of 
drugs, to other alerting systems, and to other types of phy- 
sicians or health professionals is not known and would be 
an important avenue for future research. Lastly, we 
studied physicians in one jurisdiction, and were unable to 
assess contextual factors that may modify physician 
response to alerts such as jurisdictional differences in 
malpractice litigation risk, funding for the quality of per- 
formance rather than for services alone, or the existence 
of support systems for quality improvement. 

In conclusion, to reduce the risk of psychotropic 
drug-related fall injuries, a new generation of drug alerts 
should be developed that incorporates empirical esti- 
mates of the risk of injury related to psychotropic drugs, 
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as well as patient characteristics. Eventually, these empir- 
ical risk estimates could be calculated using clinical and 
administrative data by systems capable of modelling such 
data in real time, providing earlier warnings about new 
drugs associated with increased risks of fall injuries. In 
addition, the need to incorporate more comprehensive 
feedback on the quality and outcomes of drug prescrib- 
ing for physicians that generate a high proportion of 
alerts should be evaluated in subsequent research. 
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